462 research outputs found
An Improved Baseline for Sentence-level Relation Extraction
Sentence-level relation extraction (RE) aims at identifying the relationship
between two entities in a sentence. Many efforts have been devoted to this
problem, while the best performing methods are still far from perfect. In this
paper, we revisit two problems that affect the performance of existing RE
models, namely entity representation and noisy or ill-defined labels. Our
improved baseline model, incorporated with entity representations with typed
markers, achieves an F1 of 74.6% on TACRED, significantly outperforms previous
SOTA methods. Furthermore, the presented new baseline achieves an F1 of 91.1%
on the refined Re-TACRED dataset, demonstrating that the pre-trained language
models achieve unexpectedly high performance on this task. We release our code
to the community for future research.Comment: Code available at https://github.com/wzhouad/RE_improved_baselin
Sharpness-Aware Minimization with Dynamic Reweighting
Deep neural networks are often overparameterized and may not easily achieve
model generalization. Adversarial training has shown effectiveness in improving
generalization by regularizing the change of loss on top of adversarially
chosen perturbations. The recently proposed sharpness-aware minimization (SAM)
algorithm conducts adversarial weight perturbation, encouraging the model to
converge to a flat minima. SAM finds a common adversarial weight perturbation
per-batch. Although per-instance adversarial weight perturbations are stronger
adversaries and they can potentially lead to better generalization performance,
their computational cost is very high and thus it is impossible to use
per-instance perturbations efficiently in SAM. In this paper, we tackle this
efficiency bottleneck and propose sharpness-aware minimization with dynamic
reweighting ({\delta}-SAM). Our theoretical analysis motivates that it is
possible to approach the stronger, per-instance adversarial weight
perturbations using reweighted per-batch weight perturbations. {\delta}-SAM
dynamically reweights perturbation within each batch according to the
theoretically principled weighting factors, serving as a good approximation to
per-instance perturbation. Experiments on various natural language
understanding tasks demonstrate the effectiveness of {\delta}-SAM
Context-faithful Prompting for Large Language Models
Large language models (LLMs) encode parametric knowledge about world facts
and have shown remarkable performance in knowledge-driven NLP tasks. However,
their reliance on parametric knowledge may cause them to overlook contextual
cues, leading to incorrect predictions in context-sensitive NLP tasks (e.g.,
knowledge acquisition tasks). In this paper, we seek to assess and enhance
LLMs' contextual faithfulness in two aspects: knowledge conflict and prediction
with abstention. We demonstrate that LLMs' faithfulness can be significantly
improved using carefully designed prompting strategies. In particular, we
identify opinion-based prompts and counterfactual demonstrations as the most
effective methods. Opinion-based prompts reframe the context as a narrator's
statement and inquire about the narrator's opinions, while counterfactual
demonstrations use instances containing false facts to improve faithfulness in
knowledge conflict situations. Neither technique requires additional training.
We conduct experiments on three datasets of two standard NLP tasks, machine
reading comprehension and relation extraction, and the results demonstrate
significant improvement in faithfulness to contexts. Code and data are released
at https://github.com/wzhouad/context-faithful-llm.Comment: Accepted at EMNLP 2023 Findings. Code and data are released at
https://github.com/wzhouad/context-faithful-ll
- …